How to Replay a Data Test
Many times you will want to see how a dataset plays out over time. This could be five days or five months. Using this slider the tool will automatically create training sets and profiles as well as run any rules or outliers you've put in place.
Quickly Replay 30 days of data, -br 30
Add -br to any DQCheck and replay in time order. Jan 1st, Jan 2nd, Jan 3rd...To do this we need to use the ${rd} variable that DQ provides as a run_date replacement for job control and templates. Also note that if you run from the cmdline you need to escape "$"s. So use \${rd}. If you are running from a Notebook or Java or Scala or the Rest API you do not need to escape the ${rd} variable.
./owlcheck \
-ds OWLDB2.NYSE_STOCKS3 -rd "2018-01-14" \
-lib "/opt/owl/drivers/db2/" \
-cxn db2 \
-q "select * from OWLDB2.NYSE_STOCKS where TRADE_DATE = '\${rd}'" \
-br 4Replay 4 Months of data, -br 4 -tbin MONTH
In situations where your data rolls up into Months, you may want to re-run several months of data but not a day at a time. In this case we will use -br with -tbin.
./owlcheck \
-ds OWLDB2.NYSE_STOCKS3 \
-rd "2018-01-01" \
-q "select * from OWLDB2.NYSE_STOCKS where TRADE_DATE = '\${rd}'" \
-br 4 \
-tbin MONTH \
-lib "/opt/owl/drivers/db2/" \
-cxn db2Monthly using a range for the entire Month
./owlcheck \
-ds OWLDB2.NYSE_STOCKS3 \
-rd "2018-01-01" \
-rdEnd "2018-02-01" \
-q "select * from OWLDB2.NYSE_STOCKS where TRADE_DATE >= '${rd}' and TRADE_DATE < '${rdEnd}'" \
-br 4 \
-tbin MONTH
-lib "/opt/owl/drivers/db2/" \
-cxn db2Monthly on the last day of the month
To create a back run for the last day each month, ensure your run date is the 31st of a month with 31 days in it and the back run time bin is set to MONTH. If a month included in the back run does not have 31 days, the back run job will run on the next available day. For example, if September (a month with only 30 days) is included in the scope of the back run, a back run will occur on September 30.
./owlcheck \
-ds OWLDB2.NYSE_STOCKS3 \
-rd "2024-12-31" \
-q "select * from OWLDB2.NYSE_STOCKS\
-br 4 \
-brbin MONTH
-lib "/opt/owl/drivers/db2/" \
-cxn db2